<!DOCTYPE HTML>
<!--
	Dimension by HTML5 UP
	html5up.net | @ajlkn
	Free for personal and commercial use under the CCA 3.0 license (html5up.net/license)
-->
<html>
 <head>
  <title>
   Dimension by HTML5 UP
  </title>
  <!-- <meta charset="utf-8" /> -->
  <!-- <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" /> -->
  <meta charset="utf-8"/>
  <meta content="width=device-width,initial-scale=1.0" name="viewport"/>
  <link href="../../assets/css/article.css" rel="stylesheet"/>
  <link href="https://cdn.bootcss.com/highlight.js/9.15.8/styles/github.min.css" rel="stylesheet"/>
  <noscript>
   <link href="../../assets/css/noscript.css" rel="stylesheet"/>
  </noscript>
 </head>
 <body>
  <div id="app">
  </div>
  <!-- built files will be auto injected -->
 </body>
 <body class="is-preload">
  <!-- Wrapper -->
  <div id="wrapper">
   <!-- Main -->
   <div id="main">
    <article id="article">
     <h1 id="snort">
      Snort
     </h1>
     <hr/>
     <h2 id="_1">
      关于性能的问题：
     </h2>
     <p>
      snort是一款轻量级的网络入侵检测系统。如果将其应用在大规模、大流量的网络中 大流量的网络中，snort的报警性能会非常低下。有下面的两个因素影处理性能：
     </p>
     <ul>
      <li>
       数据包捕获效率
      </li>
      <li>
       数据包分析处力效率
      </li>
     </ul>
     <h3 id="_2">
      数据包捕获效率
     </h3>
     <p>
      传统的数据包捕获流程如下(NAPI polling)：
– 网卡中断处理程序
– 网卡接收程序
– 分配skb内存
– 将其放入softnet_data队列中
– 置软中断位
– do softirq do
_softirq
– net_rx_action
– iprecv
– IP层检查数据有效性
– TCP/UDP协议处理
– 唤醒用户层进程
     </p>
     <p>
      主要观点是流量包需要先经过系统进行抓取和保存，而目前的linux系统是否能够抓取所有的数据包？这是个首要的影响条件
     </p>
     <h3 id="_3">
      数据包分析处理效率
     </h3>
     <p>
      系统在抓取数据包以后，snort会从系统中取出数据包进行处理，如那些检测规则的匹配处理之类的，而第二个影响因素就是snort内部的处理代码的问题，可能会涉及到CPU、内存、磁盘之类（磁盘好像不影响，因为数据包相当于从内存中取的）。
     </p>
     <h3 id="_4">
      自己的测试
     </h3>
     <p>
      利用一个发包的软件在15秒左右发送了120万个数据包，这些包都能引起snort的警报处理，得到的结果如下：
     </p>
     <div class="codehilite">
      <pre><span></span><code><span class="err">Packet I/O Totals:</span>
<span class="err">   Received:      1859825</span>
<span class="err">   Analyzed:       746655 ( 40.147%)</span>
<span class="err">    Dropped:      1113163 ( 37.443%)</span>
<span class="err">   Filtered:            0 (  0.000%)</span>
<span class="c">Outstanding:      1113170 ( 59.853%)</span>
<span class="c">   Injected:            0</span>
</code></pre>
     </div>
     <p>
      可以从上面看出snort接收到包是180万个（应该可以视为系统也只接收到这个数量的包吧？因为数据包是从系统中读取的，应该是读完的，那就说明现在系统是有丢包的），而只来得及分析了40%，这个部分可以是上面的数据包分析处理效率问题了。
     </p>
     <p>
      从上面的结果应该可以看出数据包捕获效率和数据包分析效率好像都有影响了，对了，下面是10秒内12万个数据的处理结果：
     </p>
     <div class="codehilite">
      <pre><span></span><code><span class="err">Packet I/O Totals:</span>
<span class="err">   Received:       222642</span>
<span class="err">   Analyzed:       222641 (100.000%)</span>
<span class="err">    Dropped:            0 (  0.000%)</span>
<span class="err">   Filtered:            0 (  0.000%)</span>
<span class="c">Outstanding:            1 (  0.000%)</span>
<span class="c">   Injected:            0</span>
</code></pre>
     </div>
     <p>
      不知道数据包为啥是22万，但多个测试下来感觉基本正常情况下大概呈现2倍的关系，也就是说上面的测试基本上丢了一些的包？还分析不过来？测试下来发现CPU在上面的测试中没有用到100%过，最高50%左右，还是数据包刚开始发送的时候，很多时候基本都是20%左右的CPU占用率，感觉是与系统的数据包数量有关吧，毕竟抓不了那么多的数据包。后面随着发包总数的增大，但发包的频率不变（基本1万1秒），抓包率好像基本维持在3/4,snort的处理率维持在50%左右（按照数据来看是这样的）
     </p>
     <h2 id="_5">
      解决方案
     </h2>
     <p>
      网上提供了一些解决方案，但还没来得及去尝试
     </p>
     <h3 id="_6">
      系统抓包方面
     </h3>
     <ul>
      <li>
       高效的数据包捕获接口--PF RING
      </li>
      <li>
       万兆网络数据包捕获--TNAPI
      </li>
      <li>
       万兆网络数据包捕获--DNA
      </li>
     </ul>
     <p>
      上面这些都是针对系统抓取数据包的，基本思想是改进原本系统的抓包流程，有一些需要配置的要求如CPU：I t l X X5675 Intel Xeon X5675（6核心，12线程）x2， 内存24G啥的
     </p>
     <h3 id="_7">
      数据包处理方面
     </h3>
     <p>
      看他们基本上都是开启多个snort的，万兆的开了12个，网上说是够用了
     </p>
     <h2 id="snort_1">
      snort过滤包的问题
     </h2>
     <p>
      前期担心数据量过大，考虑进行数据包过滤，由于UDP包在网络攻击中的作用不大，将丢弃UDP包。
     </p>
     <p>
      经过调研后决定使用snort的 -F
      <bpf>
       指定BPF过滤器。测试的数据如下，分别是过滤了TCP包和不过滤TCP包的数据处理结果和CPU使用情况
      </bpf>
     </p>
     <div class="codehilite">
      <pre><span></span><code><span class="o">#</span> <span class="err">只使用</span><span class="n">Windows实验环境中发包程序</span><span class="err">，</span><span class="mi">90</span><span class="err">线程最大速度</span>
<span class="o">#</span> <span class="err">只要</span><span class="n">TCP包</span><span class="err">，其他的过滤掉，好像发包程序发的不是</span><span class="n">tcp</span><span class="err">，是啥</span><span class="n">ARP</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">0</span><span class="p">.</span><span class="mi">3</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">10</span><span class="err">秒内处理了</span><span class="mi">6</span><span class="err">个</span>

<span class="o">#</span> <span class="err">不进行包过滤</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">29</span><span class="p">.</span><span class="mi">2</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">11</span><span class="err">秒内处理了</span><span class="mi">190851</span><span class="err">个</span>

<span class="o">#</span> <span class="err">使用第三方发送</span><span class="n">UDP包和TCP包</span><span class="err">，总共速率</span><span class="mi">31</span><span class="n">M</span>
<span class="o">#</span> <span class="err">只要</span><span class="n">TCP包</span><span class="err">，其他的过滤掉</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">31</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">287954</span> <span class="mi">11</span><span class="err">秒</span>

<span class="o">#</span> <span class="err">不进行包过滤</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">42</span><span class="p">.</span><span class="mi">3</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">649593</span> <span class="mi">12</span><span class="err">秒</span>

<span class="o">#</span> <span class="err">使用使用</span><span class="n">Windows实验环境中发包程序90线程最大速度</span><span class="err">，第三方发送</span><span class="n">TCP包</span><span class="err">，速率</span><span class="mi">31</span><span class="n">M</span>
<span class="o">#</span> <span class="err">不进行包过滤</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">39</span><span class="p">.</span><span class="mi">3</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">5845501</span> <span class="mi">25</span><span class="err">秒</span>

<span class="o">#</span> <span class="err">只要</span><span class="n">TCP包</span><span class="err">，其他的过滤掉</span>
<span class="n">CPU使用情况</span><span class="err">：</span><span class="mi">0</span><span class="p">.</span><span class="mi">1</span><span class="o">%</span>
<span class="err">处理包的数据量：</span><span class="mi">27690</span> <span class="mi">24</span><span class="err">秒（刚好是第三方发送数据包数量）</span>
</code></pre>
     </div>
     <p>
      从上面的数据的处理包的数据量可以看出使用过滤是有效的，snort对TCP协议以外的包进行了过滤。从CPU的使用情况可以看出过滤能相应的提升snort的处理性能，过滤后不进行后面的规则匹配，应该能提升snort的处理速度。
     </p>
     <h2 id="other">
      other
     </h2>
     <p>
      /usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf --daq-dir=/usr/local/lib/daq --daq pfring -i enp0s31f6
docker rm $(docker ps -a -q)
docker run -tid --net=host --name snort snort /usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf -i enp0s31f6
     </p>
     <p>
      /usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf -i enp4s0 -F /etc/snort/filter.bpf
     </p>
     <h2 id="_8">
      实验数据
     </h2>
     <h3 id="15120">
      15秒左右发送了120万个数据包
     </h3>
     <div class="codehilite">
      <pre><span></span><code><span class="n">Packet</span> <span class="n">I</span><span class="o">/</span><span class="n">O</span> <span class="n">Totals</span><span class="p">:</span>
   <span class="n">Received</span><span class="p">:</span>       <span class="mi">497600</span>
   <span class="n">Analyzed</span><span class="p">:</span>       <span class="mi">497600</span> <span class="p">(</span><span class="mi">100</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
    <span class="n">Dropped</span><span class="p">:</span>       <span class="mi">570717</span> <span class="p">(</span> <span class="mi">53</span><span class="p">.</span><span class="mi">422</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Filtered</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
<span class="n">Outstanding</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Injected</span><span class="p">:</span>            <span class="mi">0</span>

<span class="n">Packet</span> <span class="n">I</span><span class="o">/</span><span class="n">O</span> <span class="n">Totals</span><span class="p">:</span>
   <span class="n">Received</span><span class="p">:</span>       <span class="mi">540223</span>
   <span class="n">Analyzed</span><span class="p">:</span>       <span class="mi">540223</span> <span class="p">(</span><span class="mi">100</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
    <span class="n">Dropped</span><span class="p">:</span>       <span class="mi">735552</span> <span class="p">(</span> <span class="mi">57</span><span class="p">.</span><span class="mi">655</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Filtered</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
<span class="n">Outstanding</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Injected</span><span class="p">:</span>            <span class="mi">0</span>

<span class="n">Packet</span> <span class="n">I</span><span class="o">/</span><span class="n">O</span> <span class="n">Totals</span><span class="p">:</span>
   <span class="n">Received</span><span class="p">:</span>       <span class="mi">591759</span>
   <span class="n">Analyzed</span><span class="p">:</span>       <span class="mi">591759</span> <span class="p">(</span><span class="mi">100</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
    <span class="n">Dropped</span><span class="p">:</span>       <span class="mi">893606</span> <span class="p">(</span> <span class="mi">60</span><span class="p">.</span><span class="mi">161</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Filtered</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
<span class="n">Outstanding</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Injected</span><span class="p">:</span>            <span class="mi">0</span>

<span class="n">Packet</span> <span class="n">I</span><span class="o">/</span><span class="n">O</span> <span class="n">Totals</span><span class="p">:</span>
   <span class="n">Received</span><span class="p">:</span>       <span class="mi">675657</span>
   <span class="n">Analyzed</span><span class="p">:</span>       <span class="mi">675657</span> <span class="p">(</span><span class="mi">100</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
    <span class="n">Dropped</span><span class="p">:</span>      <span class="mi">1012010</span> <span class="p">(</span> <span class="mi">59</span><span class="p">.</span><span class="mi">965</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Filtered</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
<span class="n">Outstanding</span><span class="p">:</span>            <span class="mi">0</span> <span class="p">(</span>  <span class="mi">0</span><span class="p">.</span><span class="mi">000</span><span class="o">%</span><span class="p">)</span>
   <span class="n">Injected</span><span class="p">:</span>            <span class="mi">0</span>
</code></pre>
     </div>
     <div class="codehilite">
      <pre><span></span><code><span class="err">Total packets processed:              176719</span>
<span class="err">Total packets processed:              174065</span>
<span class="err">1192681</span>
</code></pre>
     </div>
     <h2 id="other_1">
      other
     </h2>
     <p>
      /usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf --daq-dir=/usr/local/lib/daq --daq pfring -i enp0s31f6
/usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf -i enp0s31f6
docker rm $(docker ps -a -q)
docker run -tid --net=host --name snort snort /usr/local/bin/snort -A console -u snort -g snort -c /etc/snort/snort.conf -i enp0s31f6
     </p>
     <h2 id="_9">
      参考链接
     </h2>
     <ul>
      <li>
       <a href="http://sourcedb.ict.cas.cn/cn/ictthesis/200907/P020090722605372300999.pdf">
        关于Snort的抓包原理及性能改进(可能版本较老，不知现在的版本如何)
       </a>
      </li>
      <li>
       <a href="http://www.owasp.org.cn/OWASP_Events/download/snort.pdf">
        SNORT原理简介与优化及GNORT初探
       </a>
      </li>
      <li>
       <a href="https://blog.csdn.net/zhwei_87/article/details/39643309">
        Linux配置hiredis
       </a>
      </li>
      <li>
       <a href="https://github.com/redis/hiredis/releases">
        hiredis 下载地址
       </a>
      </li>
      <li>
       <a href="http://blog.51cto.com/yuzwei/1716803">
        PF_RING安装与使用总结
       </a>
      </li>
      <li>
       <a href="https://blog.csdn.net/u010390063/article/details/79078756">
        高性能流量生成工具trafgen(DDoS模拟)
       </a>
      </li>
      <li>
       <a href="https://snort-org-site.s3.amazonaws.com/production/document_files/files/000/000/014/original/PF_RING_Snort_Inline_Instructions_daq_062.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAIXACIED2SPMSC7GA%2F20180725%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20180725T122112Z&amp;X-Amz-Expires=172800&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=f782832c1784b438e7e3425a256a5186f8ccdfbb1840179df775411e7a7668f4">
        Inline Snort multiprocessing with PF_RING(尝试成功的snort结合pf_ring版本)
       </a>
      </li>
      <li>
       <a href="https://www.ntop.org/guides/pf_ring/get_started/git_installation.html#">
        Installing from GIT
       </a>
      </li>
      <li>
       <a href="https://www.ntop.org/guides/pf_ring/thirdparty/snort-daq.html">
        Using Snort with PF_RING
       </a>
      </li>
      <li>
       <a href="https://www.snort.org/downloads#snort-downloads">
        snort download
       </a>
      </li>
      <li>
       <a href="https://snort-org-site.s3.amazonaws.com/production/document_files/files/000/000/122/original/Snort_2.9.9.x_on_Ubuntu_14-16.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAIXACIED2SPMSC7GA%2F20180725%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20180725T091034Z&amp;X-Amz-Expires=172800&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=fa38cc81bc1cedaee8b4696643e31bff92ae9e7fe46d23025c9edc9f8156b7fb">
        snort 2.9 docment
       </a>
      </li>
      <li>
       <a href="https://snort-org-site.s3.amazonaws.com/production/document_files/files/000/000/136/original/Snort_3_on_CentOS_7.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAIXACIED2SPMSC7GA%2F20180725%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20180725T110233Z&amp;X-Amz-Expires=172800&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=6e23ca85dfff5c03df79af70fef0178ace629137c3a641c8d48496b5d43d445c">
        snort 3.0 docment
       </a>
      </li>
      <li>
       <a href="https://www.snort.org/documents">
        snort Documents
       </a>
      </li>
      <li>
       <a href="http://www.freebuf.com/sectool/36545.html">
        免费DDOS攻击测试工具大合集
       </a>
      </li>
      <li>
       <a href="https://juejin.im/entry/5aa792586fb9a028d936d5e9">
        推荐两款网络攻击测试工具
       </a>
      </li>
      <li>
       <a href="https://cloud.tencent.com/document/product/213/11460">
        网络性能测试
       </a>
      </li>
      <li>
       <a href="http://blog.51cto.com/wushank/1585927">
        linux下的CPU、内存、IO、网络的压力测试
       </a>
      </li>
      <li>
       <a href="https://blog.csdn.net/zhangsheng_1992/article/details/52806477">
        liunx检测上下行带宽及丢包率
       </a>
      </li>
      <li>
       <a href="http://linuxtools-rst.readthedocs.io/zh_CN/latest/tool/crontab.html">
        Docs » 工具参考篇 » 19. crontab 定时任务
       </a>
      </li>
      <li>
       <a href="https://www.jianshu.com/p/838db0269fd0">
        Linux之crontab定时任务
       </a>
      </li>
      <li>
       <a href="https://blog.csdn.net/jack237/article/details/6899465">
        Snort 命令参数详解
       </a>
      </li>
      <li>
       <a href="https://blog.csdn.net/luguifang2011/article/details/72953917">
        BPF过滤规则及tcpdump命令详解
       </a>
      </li>
      <li>
       <a href="https://www.cisco.com/c/zh_cn/support/docs/security/firesight-management-center/118090-configure-sourcefire-00.html">
        SNORT_BPF变量的配置在防御中心的
       </a>
      </li>
     </ul>
    </article>
   </div>
   <!-- Footer -->
   <footer id="footer">
    <p class="copyright">
     © Untitled. Design:
     <a href="https://html5up.net">
      HTML5 UP
     </a>
     .
    </p>
   </footer>
  </div>
  <!-- BG -->
  <div id="bg">
  </div>
  <!-- Scripts -->
  <script src="../assets/js/jquery.min.js">
  </script>
  <script src="../assets/js/browser.min.js">
  </script>
  <script src="../assets/js/breakpoints.min.js">
  </script>
  <script src="../assets/js/util.js">
  </script>
  <script src="../assets/js/main.js">
  </script>
 </body>
</html>
